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(54) "Device with a human-machine interface" 

(57) A device with a human-machine interlace for 
use as a user control interface, which temporarily 
makes an indication of the function of a control organ 
perceptible, when the user begins to actuate it. but has 
not yet definitively actuated it within a predetermined 
period of time. The device with a human-machine inter- 
face helps the user to find the right control organ, or 
warns about dangerous control organs, without burden- 
ing the confident user. The indication is disclosed only if 
definitive actuation of the control organ is not detected 
within a predetermined period of time after detection of 
user action to begin actuation. 
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Description 



[0001] The invention as defined in the appended 
claims relates to a device with a human-machine inter- 
face, the device comprising an actuable control organ 
for selectably controlling a function of the system, the 
device comprising means for detecting a user action 
representative for physically impending actuation of the 
control organ, the device having means for, upon said 
detecting, temporarily disclosing via output means, an 
indication of the selected function to the user. 
[0002] Such a device is known from US patent No. 
4,390,861, according to which the indication is visually 
displayed. If the indication corresponds to the one 
desired by the user, the user is able to proceed with the 
coherent movement and actuate the control organ, If 
the indicated function is not desired, the user is able not 
to proceed to the actual actuation stage, thus averting 
execution of an undesired function. 
[0003] From the German Offenlegungsschrift DE 
3700913 a user control interface for a computer device 
is known, in which the control organs are keyboard 
keys, which can be actuated by pressing. Images corre- 
sponding to the keys are displayed together on a display 
screen, providing an identification of the function asso- 
ciated with each key. 

[0004] From N.T.I.S. Technical notes vol B. no.6 
Springfield Virginia. USA page 672 a device is know 
that provides an audible indication. 
[0005] Amongst others, it is an object of the invention 
to provide for a device in which the user can avoid being 
burdened by the cfisplay of the indication of the selected 
function. 

[0006] According to the invention the device with a 
user control interface according to the invention, is 
arranged for effecting said disclosing only when, after 
detecting said use- action, actuation of the control organ 
is not detected within a predetermined period of time. In 
this way the user can avoid being burdened by the dis- 
closing through acting quickly, for example within a pre- 
determined time of a few tenths of a second. This is 
convenient for example for experienced user, or for a 
user which is already confident about the function of the 
control organ after trying a neighboring control organ. 
[0007] In an embodiment of the device with a user 
control interface according to the invention it is arranged 
for disabling the disclosing when actuation of the control 
organ is detected. In this way, once the decision to exe- 
cute the control function is effected, the user is not bur- 
dened longer than necessary by the disclosing. 
[0008] In another embodiment of the device with a 
user control interface according to the invention, it is 
arranged for disclosing a further indication of the func- 
tion when actuation of the control organ is not detected 
the within a further predetermined period of time after 
detecting said user action. Thus a hesitant user can be 
helped, for example by providing help information 
explaining the function of the control organ; a useful 



time interval before providing help information is in the 
order of several seconds. 

[0009] These and other advantageous aspects of the 
device with a human-machine interface according to the 
s invention will be illustrated using the following Figures. 

Figures 1a, 1b. 1c show an embodiment of the suc- 
cessive types of actuation of control organs in an 
device with a human-machine interface 
Figure 2 shows a schematic diagram of an imple- 
mentation of the device with a human-machine 
interface according to the invention 
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[0010] In Figures 1a, 1b, 1c an example of an embod- 
is iment a group 10 of control organs 11, 12, 13 for a 
device with a user control interface is shown. By way of 
example, the control organs are shown as push buttons 
which can be pushed by a finger 14. Conventionally, to 
select one button 12 from the plurality 1 0. the user must 
so look at the control organs to ensure that the finger is 
placed on the desired button 12 from the plurality 10. 
[001 1] In vehicles, like cars, (motor)bicycles etcetera 
in which the user at the same time is responsible for 
controlling for example the steering wheel and the 
25 brakes, it is undesirable that the user should divert the 
eyes to look at the plurality of control organs 1 0. Prefer- 
ably, this should even be made impossible, by placing 
the control organs outside the normal line of sight from 
the drivers seat, for example under a cover. Yet for 
30 instance to control a car radio, the user must reach for 
the control organs; pushing one button reached blindly, 
that is, without looking at it first, is likely to cause errors' 
which may be hard to correct -for example in the case of 
detuning the car radio- or costly -for example in dialing 
35 a wrong telephone number on a car phone. An error 
may even be damaging or dangerous in case the control 
button pressed erroneously concerns the driving of the 
car. 

[0012] To prevent such errors, the device wfth a user 
40 control interface according to the invention provides, 
upon detection of a user action representative of 
impending actuation of a control organ, for an indication 
of the function of that control organ in such a way that 
the user can still prevent the function from being 
45 effected by not pursuing the action beyond the stage 
that lead the interface to conclude that actuation was 
physically impending. To prevent the user from being 
unnecessarily distracted by the mechanism for giving 
this indication, the indication is made available via a 
so standard output interface used for different types of 
messages. It is made available temporarily, only for a 
limited time after detection of impending actuation of the 
control organ, that is, it is made perceptible to the user 
only at an attempt to actuate a function. This minimizes 
55 distraction to the user, as is essential in a vehicle. 
[0013] Of course a device with a human machine 
interface without normally perceptible control organs 
will be useful in other environments than vehicles as 
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well. 

[001 4] To achieve such an interface, the idea is that an 
actuating action by the user will have successive 
stages, the interface effecting the function of the control 
organ only in the final stage, and in earlier stages pro- 
viding indication of the function of the control organ 
selected. Thus, the user may avoid effecting the func- 
tion by selectively breaking off the action before the final 
stage is reached. 

[0015] To illustrate the principle, in Figures 1a, 1b, 1c, 
the successive stages are triggered by successive dis- 
tances of approach of the finger 14 to the control organ 
12. The distance of the finger may for example be 
sensed by a proximity sensor operating by means of 
light, sound or heat. The stage of physically impending 
actuation occurs in Figure 1 a when the finger 1 4 passes 
the border indicated by line 15; when the device with a 
human-machine interface detects this, it provides an 
indication of the function associated with the control 
organ 12. Actual actuation is effected in Figure 1b when 
the finger 14 passes the border 17; this causes the 
device with a user control interface to effect the function 
associated with the control organ 12. The impending 
actuation is a physical precondition for the user to per- 
form actuation. 

[0016] It may be convenient to provide means for 
detecting two or more stages of actuation of all control 
organs. However, often it will be more attractive to do so 
for control organs corresponding to important functions 
only or for functions which are not easily reversible. This 
may be more convenient for economic reasons or for 
giving a higher alerting effect to the indication. 
[001 7] The control organs are often coherently placed 
in groups of control organs which control related func- 
tions. For example in an automotive environment the 
controls for heating are placed together, the controls for 
the car radio are placed together. It may be desirable to 
report first the function of the group and then the func- 
tion of a specific control organ before effecting that func- 
tion. For example to help the user to find a desired 
group more quicWy by giving a coarse indication of the 
position of the finger. A way to achieve this is illustrated 
in Figure 1c, where an intermediate actuation stage is 
detected when the finger 1 4 passes the border 16. In an 
device with a human-machine interface comprising 
three borders 15, 16. 1 7, the first border 1 5 might trigger 
function group indication, the second border 16, specific 
control organ information and the third border 17 might 
cause the function to be effected. 
[0018] Function group indication is also particularly 
useful when the function of a group of control organs 
may be reassigned as a group, for example when the 
same control organs selectably either control a car tele- 
phone or a car radio. In this case the function group indi- 
cation serves to warn the user when the function group 
must be reassigned. 

[0019] Although the invention was discussed above 
with respect to pushbuttons, the invention is applicable 
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to any kind of control organs, such as for example 
potentiometers, either of the rotating or of the sliding 
variety. 

[0020] It will be appreciated that there exist various 
5 possibilities for implementing detection mechanism for 
actuation and impending actuation of control organs. 
The only essential point is that the mechanisms are 
arranged for detecting more than one physical stage in 
an action that may lead to actuation, and that these 
10 stages are arranged in such a way that, when used by a 
human user, successive stages are physically precondi- 
tions for later stages, but that the later stage may also 
be foregone if the user decides to. 
[0021] In its conceptually simplest form this is 
is achieved by providing individual control organs with at 
least two sensing mechanisms, one for detecting the 
physically impending actuation, another for detecting 
the actual actuation, the one being sensitive to an ear- 
lier stage of approach of a finger than the other. How- 
20 ever, for the purpose of the invention, the there need not 
be a physical relation between means for detecting 
actuation, physically impending actuation an a particu- 
lar control organ: 

25 1) the detecting mechanisms for actuation and 
impending actuation may be separated physically, 
that is. be located at geometrically different loca- 
tions, 

2) the detecting mechanisms may be shared 
30 among control organs, 

3) several control organs may be provided as a sin- 
gle physical device. 

To illustrate these possibilities: 1). for example rf a push 
35 button is used for detecting the actuation, and an optical 
sensor is used for detecting the impending actuation, 
there is no need to place the pushbutton and the optical 
sensor together, as long as they sense the finger in the 
same region. To illustrate 2), using touch sensitive but- 
40 tons for detecting the impending actuation (that is, for 
example capacitive sensors, which are actuated by 
touching rather than by exerting force) and a force sen- 
sor to detect the second actuation, the force sensor may 
be shared by several control organs since touching of a 
45 touch sensor is sufficient to determine which control 
organ is to receive the actual actuation. This has the 
advantage that it is guaranteed that both the impending 
and actual actuation will automatically correspond to the 
same control organ. To illustrate 3), one may use a corn- 
so mon touch sensitive surface provided with means to 
determine the coordinates of a point where it is touched; 
the coordinate values are used select one virtual control 
organ and detect the impending or actual actuation for 
this control organ; in this way one sensor serves for a 
55 plurality of control organs. Since according to the inven- 
tion the functions of the control organs are identified 
with separate messages, such a touch tablet would 
require no further distinguishing marks to identify the 
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control organs. Thus, an automobile, or indeed any con- 
trol interface in any application outside a vehicle, could 
be provided without visible control organs. 
[0022] There is also an ample choice of sensors. One 
implementation uses a proximity sensor for impending s 
actuation (the presence of a proximate object being 
detected using reflected or absorbed light, sound, elec- 
tric or magnetic fields etc.) and a push button for detect- 
ing the actual actuation. Alternatively, the control organ 
may be sensitive independently to pushing and touching io 
(without pushing) so as to detect the impending and 
actual actuation as a consequence of pushing and 
touching respectively. In another embodiment, the con- 
trol organ is a normal control organ, for example a push- 
button, which serves to detect the impending actuation; w 
the actual actuation is detected if the button is kept 
pushed during a predetermined period of time (that is, 
long enough for the user to realize a mistake from the 
indication triggered by the first actuation). 
[0023] In any case, several actuation signals, corre- 20 
sponding to different stages of actuation will have to be 
derived for the control organ. 
[0024] Figure 2 shows an embodiment of an device 
with a human-machine interlace according to the inven- 
tion. For simplicity, only one control organ 12 is shown, 25 
which has outputs 31, 32, 33 for three signals corre- 
sponding to detection of the first and second stages of 
impending actuation and the actual actuation respec- 
tively (the second stage being optional). In the embodi- 
ment, the first detection output is coupled to a sound 30 
signal generation circuit 22, which in turn is coupled to a 
loudspeaker 20. When the impending actuation is 
detected, for example after crossing the line 15 in Figure 
1a, the function of the control organ can thus be identi- 
fied via the loudspeaker. In a vehicle, the use of such an 35 
audible indication has the advantage that it does not 
distract the eyes of the user the from the road, and that 
no distracting signals for the user are present until a 
control organ is almost actuated. Preferably, the signal 
generator 22 produces a spoken message, for example 40 
by replaying a recorded message or using a speech 
synthesizer, but other signals, such as warning tones 
which identify functions as dangerous may also be use- 
ful. 

[0025] Alternatively, one may use a display driver 22 45 
and a head up display screen for projecting the informa- 
tion on the windscreen upon detection of impending 
actuation. 

[0026] Another output 32 of the control organ 1 2 is for 
signaling that the actual actuation is detected, which will 50 
effect the function of the control organ. The effecting is 
symbolized in Figure 2 by connection of the output 32 to 
an activating input of a subsystem 21 for function execu- 
tion. 

[0027] Several elaborations to the device with a 55 
human-machine interlace, which are not essential to its 
functioning, but may be added to give substantial 
improvements to its effectiveness in minimizing the 



amount of unnecessary distraction of the users atten- 
tion are also shown in Figure 2 and will now be dis- 
cussed. 

[0028] First of all, it may be convenient to interrupt the 
indication of a function once the function is activated by 
effecting the actual actuation. This is symbolized in Fig- 
ure 2 by a switch 23 controlled by the output for the sec- 
ond mode 32 between the signal generator and the 
loudspeaker. In this way, unnecessary messages are 
cut short; it may even be convenient to give a slight 
delay (of the order of a few tenths of a second) to the 
signal generation 22 in order to enable the unhesitating 
user to avoid any signaling at all. 
[0029] Another addition is a second function report 
generator 26, triggered by detection of a second stage 
in the action leading up to actuation by the second stage 
detection output 33. Such stages may for example be 
detected by detecting a finger at different decreasing 
distances to the control organ, but any combination of 
the detection mechanisms described above may be 
used, as long as one mechanism will trigger at an ear- 
lier stage of the user action than the other. In combina- 
tion with the first function report generator 22 the first 
and second stage detection may be used to provide a 
first function group indication after detection of the first 
stage of the action and secondly a specific function indi- 
cation after detecting the second stage of the action. It 
may be convenient to arrange the device with a user 
control interface to interrupt the group information if the 
user activates the specific information. 
[0030] Furthermore, it may be convenient to provide 
for additional help information for an extremely hesitat- 
ing user, by allowing an impending actuation signal to 
pass though a time duration thresholding circuit 24 for 
triggering a help information generating circuit. In this 
way, when the state of impending actuation persists, but 
is not followed up by actuation within, for example, a few 
seconds, additional help information can be reported 
via the loudspeaker 25. 

[0031 ] Although Figure 2 has been drawn in the form 
of a schematic diagram, isolating several functions in 
separate boxes, it will be appreciated that the part of the 
interface providing the links between the control organ 
1 2 and the output 20 can also be implemented as a pro- 
gram in a computer system. 

Claims 

1. A device with a human-machine interface, the 
device comprising an actuable control organ for 
selectably controlling a function of the system, the 
device comprising means for detecting a user 
action representative for physically impending actu- 
ation of the control organ, the device having means 
for, upon said detecting, temporarily disclosing via 
output means, an indication of the selected function 
to the user, said disclosing being effected only 
when, after detecting said user action, actuation of 
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the control organ is not detected within a predeter- 
mined period of time. 

A device according to Claims 1, characterized, in 
that it is arranged for disclosing a further indication s 
of the function when actuation of the control organ 
is not detected the within a further predetermined 
period of time after detecting said user action. 

A device accorcfing to Claim 1 or 2, characterized, 10 
in that it is arranged for disabling the disclosing 
when actuation of the control organ is detected. 
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(54) "Device with a human-machine interface" 

(57) A device with a human-machine interlace for 
use as a user control interface, which temporarily 
makes an indication of the function of a control organ 
perceptible, when the user begins to actuate it, but has 
not yet definitively actuated it within a predetermined 
period of time. The device with a human-machine inter- 
face helps the user to find the right control organ, or 
warns about dangerous control organs, without burden- 
ing the confident user. The indication is disclosed only if 
definitive actuation of the control organ is not detected 
within a predetermined period of time after detection of 
user action to begin actuation. 
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